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The statistical theory of stationary processes has wide applications in the analysis of 
radio wave propagation data. In this paper, assuming the knowledge of the basic concepts 
of probability theory on the part of the reader, characteristics of stationary processes such as 
CO variance and spectral density functions have been developed, problems of estimating these 
characteristics have been tackled, and numerous examples have been worked out to illus- 
trate the theory. 



1. Introduction 

The indeterininacy in science, in mieroseopic as 
well as macrocopic physical processes, is gradually 
replacing the concept of '^a cause and its effect'' 
by the concept of ^'a cause and the probability 
distribution of its possible effects." We no longer 
ask the questions: will this signal be received? 
will a meteor be observed during this time interval? 
Instead we ask: what is the probability that this 
signal will be received? what is the probability 
that during this time interval at least one meteor 
will be observed? 

In radio science the probability theory aiul 
statistical methods have wide applications, both in 
the development of theories and in the analysis of 
data. Scattering of radio waves from an irregular 
surface, propagation of electromagnetic field in an 
inhomogeneous iiiodium, reflection of radio signals 
from meteor trails, the interaction of solar and cosmic 
energy with the earth's atmosphere, and many otlu^' 
such processes require theoretical models based on 
some kind of probability mechanism. Some models 
for these phenomena have alrady been proposed (to 
quote a few examples: Ra3deigh [1899, 1919], Rice 
[1951], Isakovich [1952], Hoffman [1955,^ 1959], 
Wheelon [I960]), but essentially the field is wide 
open. The real challenge is to develop a sufficient 
number of stochastic models which would provide 
proper methods of analyzing mountains of data, 
which have already been collected, and point out 
ways for further meaningful experiments. In the 
absence of satisfactory theories, from which deduc- 
tions can be made, we can only rely on ad hoc 
inferential hypotheses based on the statistical 
analysis of the data. Hence we turn to the question : 
what statistical methods can be used profitably to 
reduce the data into a few meaningful numbers? 
We find that the statistical theory of stationary 
processes, if correctly applied, has been and can be 
of great use. We shall, therefore, confine our study 
to the characteristics of stationary processes and 
efficient procedures for estimating these character- 
istics. The knowledge of the basic concepts of 



probability theory will be assumed on the part of 
the reader. 

2. Definitions 

A random process, {X{t)], is a function of a 
parameter t, such that for each value of t, X{t) is a 
random variable. We will refer to t as time, although 
it may be an arbitrary parameter. If t takes on 
only discrete values, say . . ., —2, —1, 0, 1,2,. . ., 
the process, { . . ., X(-2), X{-\), X(0), X{1), 
X{2), . . . } is called a discrete-time process; and if i 
takes on values in a continuum, say — oo<^<oo^ 
{X{t), —co<t<co] is called a contvnuous-tirne proc- 
ess. Thus, for example, if we observe the hourly 
median value, M{t), of the envelope, X(t), of the 
received signal, {M{t)} is a discrete-time and {X(t)} 
is a continuous-time process. An observed record 
of {X(t)}, written without the curly brackets as 
X(f), is called a sample function. The curly brackets 
simply denote that there are infinitely many possible 
sample functions which constitute the random 
process. If there is one and only one possible 
sample function, then the process is not random but 
deterministic. 

What we want to Imow is the joint probability 
distribution of the random variables ^(^i), . . ., 
X{tn) for arbitrary n, ti, . . ., in. In its generality 
it is an impossible task, as it requires infinitely many 
sample functions observed over infinite time inter- 
vals. Thus we cannot proceed further with the 
analysis without assuming certain structure for the 
process. 

A random process, {X{t)], is called weakly station- 
ary if, for all t and s, 

EX'{t)<:oo;EX(t)=EX(0); 

EX{t)X{t+s) =EX{0)X{s) . 

Here, if X is a random variable with probability 
(X<x)=P{x), EX stands for the mean value of X, 
i.e., 
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EX 



=1 



xdP{x). 



A random process, {X(t)], is called strictly statidn- 
ary if the joint distribution of X(ti-\-s), . . ., 
X(tn-\-s) is identical with the joint distribution of 
X(ti), . . ., X(tn)^ for every n, ti, . . ., tn and s. 

A weakly or strictly stationary process, {X(t)}, is 
called ergodic in respect to a function g, if, for almost 
all sample functions, X(t), 



Eg(X(t))==Yim ^ C g{X{t- 



-s))ds. 



If for any arbitrary function g, such that Eg{X{t)) 
exists, the relation given above holds, then the process 
is called, simply, ergodic. 



3. Stationary Processes 

In general, a record of some aspect of radio propa- 
gation cannot be considered as a sample function 
frorai a stationary process. For example, let Xif), 
^=1, 2, . . ., denote the hourly median value of 
the critical frequency, /o^2, of the F2 layer of the 
ionosphere observed at Washington, D.C. From a 
priori considerations we would expect a diurnal, a 
seasonal, and a ten- or eleven-year cycle in the rec- 
ord, the last mentioned cycle corresponding to the 
sunspots cycle. However, if we eliminate these cycles 
by the least-squares fit we may reasonably assume 
that the residuals constitute a sample function of a 
stationary process. As we shall see later, theoreti- 
cally it may be admissible to consider even a record 
with discernible periodicities in it as a sample func- 
tion from a stationary process, but in a single record 
it is always advisable to remove such discernible 
cycles, and also the low-frequency part which, due to 
the limited extent of data, appears as a trend, before 
spectral analysis of the data. (For proper regression 
analysis refer to Siddiqui [I960].) 

3.1. Continuous-Time Processes 

Let [X{t)] be a continuous-time real weakly sta- 
tionary process with 

EX{t)=Q; EX{t)X(t+s)=y{s)=y{~s). (3.1) 

The question of estimating EX(t) = ii, when /x is not 
assumed to be zero, will be taken up in the next 
section. 

The spectral representation of a weakly stationary 
process with mean zero, suggested by Cramer [1940, 
1942], is a powerful method of understanding the 
characteristics of the process. 

If /i and I2 are two disjoint intervals on a real line, 
a random set function z{I) is called orthogonal if 



Here z denotes the complex conjugate of z. Let us 



write dz(f) for z(df). Cramer shows that A^^ has 
the spectral representation 

^^{t)=r e'^'^MzU). (3.2) 

•/ — 00 

where z(J) is an orthogonal set function with 

E\dz (/) ^^G{f+df) - G{j) =dG{f) , #>0,, (3.3) 

and 6(f), called the spectral distribution function, is a 
nondecreasing function with 



(?(-co)=0, G{oo)=y(0). 

Now, X{t) is real, hence 



(3.4) 



y{s)=EX(t)X{t+s)=EXit)Xit+s) 



•/ — C» »./ — 

= f" e^'"-V(?(/), 



e''^'^^-f'^'^-'-'fUh{j')dz{f') 



(3.5) 



from the orthogonahty of z{j). In (3.2) and (3.5) 
we have admitted negative as well as positive fre- 
quencies. It is possible to develop the theory in 
terms of positive frequencies only, in which case we 
will use cosine function for transformation rather 
than the exponential function. Obviously, expo- 
nential functions are much easier to work with than 
trigonometric functions. Hence, we will retain the 
representation as given above. 

The integrals (3.2), (3.5), and others which will 
appear below should be interpreted in the Stieltjes 
sense. Thus, for instance, if G(j) is a purely step 
function with steps of sizes g'o, g[, ^2, • . ., at 
/=0, ±/i, ±f2, . . ., respectively, (3.5) is to be 
interpreted as 

y{s)=go+f2gi(e'^'^''+e-'^'^^') 

k=l 



=go+2^g', cos (27rf,s). (3.5a) 

k=l 

g'j, is called the spectral mass at the frequencies 1 
±/a. In this case 



X{t)==^ao+^{a, cos27r/,/ + 6,sin27r/,0, (3.2a) 

since dz(f) is also a step function. Here 

ao=dz(0), aic=dz(Jjc)+dz(—fjc), bjc=-idz(fjc)—idz(—ff,). 



Noting that dz{—fk)=dz(J'jc) = (ak+ibjc)/2, and re- 
membering the orthogonality property of z(J), 
and that E X{t)=(}j we have 
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Ea,=-Ebk=0, all k;Ea,bj=0, all kj; Eal=--(jo, 

Eal=Ebl=2(f„ Ea,(ij=Eb,hj^O, k^j. (3.6) 

If, on the other hand, G(f) is diiierentiable with 
6'{j) = !l{f),andg(-f)=g(J): (3.5) becoiues 



y{s)- 



= J" e''<''g{J)dJ=2J\'os (2,r/,s')fif(/)r//, 



(3.5b) 



In this case, G'{J) = (j(f) is called the spectral density 
function of the process. More generally 

G(J) = G,(J) + G,(J), 

where Gi{f) is purely a step funetion, and G^if) 
admits the spectral density (j2(f)' Thus, the more 
general representations of 7(.s') and A'(/) are tlie 
following : 

y{s)=(lo+f: 2r/; COS (27r/,5)+2 f^os {2Tfs)g,{f)(//\ 

k = i Jo 

(3.5c) 



Xit)=a,+J2 (ot COS 2Tf,t + b, sill 27r/',0 



J. 



e'^^'^'dz-.W), ('^•^<0 



where E\dz2{j)?=(f2{i)([i'' and a,), a„, 6^, not oidy 
satisfy (3.6) but also are uncorrelated with dz2(J). 
It may be noted that, since a^, a^, b^ are independent 
of t, in a single sample function they appear as 
constants; hence the desirability of estimating these 
constants by the l(\nst -squares method as discussed 
earlier in the section. 

From here onwards we shall assume that by proper 
regression analysis G^(j) is elinnnated so that G{i) 
is differentiable, and the i'(^i)resentation (3. 5b) holds. 

If 



J - CO Jo 

(3.5h) can be inverted to give 



(3.7) 



Sf(-/) = 



= ;/(/)= J" e---'f'y{s)ds=^2^^ cos {2iTJs)y{s)ds, 



(3.8) 



and g(f) is continuous everywhere. Thus a sufficient 
condition for the continuity of g(f) is that \y(s)\ be 
integrable. 

It is not sufficient, however, that oidy y{s) be 
integrable, for example, if 

sin 27rBs ... 
y{s)= J By\}. 

ITS 

f/(/)=:l, if I/I <7?;0, otherwise, 
which is discontinuous at/=±i^. It may be noted 
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that, by definition, |^/(/)| is integrable. In fact 

k/a)hK/).and 



,f: 



^a>//=^(-)-7(o)<< 



hence, if a continuous-time process possesses a sjMctral 
density function, its covariance function y(s) is 
continuous e very where . 

Many a time we wish to obtain th(^ i*(^lationship 
between the spectral density functions of two proc- 
esses {X(t)} and {y{t)}, wiiich are related through 
some linear differential or integral ecpiation. 'I'hus, 
for example, the frequency of a signal is tlu^ deriva- 
tive of its phase. 

Let {.Y(0} be a weakly stationary process, \V{t) 
a rc^al intc^grable function, and \c{ 

Y(t)= r \V(:u)X{t-u)du= r W{t-u)X(u)du. 

t/ — CO *./ — CO 

(3.9) 

\\\1) is called the linear filter, X{t) the input and 
rU) the output of the filter Wit). If Wit) = {) for 
<<(), \\'(t) is a ])hvsica!Iv realizal)ic filler. 
Let 

A'(0=J_jr'"Vz,(./), 

/» CO 

lF*(/)= I e-'""W(u)du. 
From (3.9), we ohlain 

Yit)=r \V*{J)e''"^dzAJ). 

tJ — CO 

Thus }'(0 Inis the representation 

Y{t)= f" e-''"^dz„U), 

t/ — 00 



(3.10) 



where 



dz,U)=W*if)ds,(f}. 



Hence, by (3.3), 



9AJ) = \W*U)\%(.f). 



(3.11) 



(3.12) 



where g„( /') and g^{ f) are the spectral dcTisitv functions 
of {Y{t)} and {X(t)}, respectively. Thus {¥{1)} is 
also a weakly stationary process. It is to be noted 
that the integrability of W{t) is an essential condition 
for (3.12) to hold. Also, we require that (ji,{f) be 
integrable. 

Let us now consider 

U it)=a,X'''\t)+a,^,X'''~'\t)+ . . .+a.oX(0, 

(3.13) 



where X^^\t)=d^X/dt^. Again, using the spectral 
representation of {X(t)} process, we obtain 



guU)= 



S {2^ifya\ gAJ). (3.14) 

j=o I 



Of course, {U{t)} will be defined if and only if 
gu(f) is integrable, i.e., when 7w(0)< oo . 

A particular case of interest is obtained from (3.13) 
by setting aj,= l, aj=0, j^p, so that U{t) = X^^\t). 
In this case 



r/,(/) = (27r)^-/^^,(/). 



(3.15) 



Thus the process {X(t)} is differentiahle p times if 
and only if the 2pth (hence every lower order) moment 
of the sjJectral density function , gx(f), exists. 
Differentiating yx{t), 2p times in the relation 



-i 



we obtain, from (3.15) 

7u{t) = (-iryi''\t). 



(3.16) 



Thus, equivalently , the process {X(t)} will he differ- 
entiable p times if and only ij 72;(t) is diferentiable 
2p times. 

Since for any^process {X{t)}, |7x(OI <T:r(0), yx(t) 
attains its maxunum at ^ = 0. Hence, if y^it) is 
differentiable 2/> times, we must have 

0^^a)(0)=7i3)(o)= . . . _7(2.-i)(o)^ (3.17) 
since each of 

is a CO variance function of a stationary process. 
Furthermore, EX(t)X^^(t+s)=y^^^(s); hence X(t) 
and X^'^^~'^^(t), j=l, . . ., p, will be uncorrelated. 

3.2. Discrete-Time Processes 

It is a common practice to observe a process at 
equal intervals of time even though the process may 
be a continuous-time process. Let t be measured in 
seconds so that / is measured in cycles per second. 
Let the spacing between observations be h seconds, 
so that the derived process is {X(kh)}, k= . . ., 
— 1, 0, 1, . . ., with covariance function yj.=y(kh), 
k=0, ±1, ±2, ... . From (3.5b) 

y,=y {kh)= r e'^'^''g{f)dj 

»/ — c» 

00 /»(2r-f-l)/(2/»^ 

r = -co J (2r-l)/(,2h) 



Jl/(2/j) 
-1/(2/1) 



'g.iDdJ, 



(3.5d) 



where 

+ .^/(/-0}^ -1/(2A)</<1/(2A). (3.19) 

The frequencies /±r/A, r=l,2, . . ., which become 
indistinguishable from the frequenc.y /, are called 
aliases to/. In case 2|7^|<;oo ^ (3.5d) can be inverted 
to give 



^i(/)=7o+2g7. cos (2x/M),-l/(2A)</<l/(2A.). 

(3.8d) 

Of course, if we start with a discrete-time process, 
we ma}^ conveniently set h=l, and obtain 



f 1/2 



COS (27r//,-) i/i (/)(//, ^=0, ±1, ±2, . . ., 



</'i(./)=To+2 S 7* cos i2^fk), -l/2< /•< 1/2. 
Also, the spectral representation of {X(k)} is 

J -1/2 

VsAt\xE\dz,{j)\'=<l,{j)df. 

Relations corresponding to (3.12) and (3.14) can 
easily be obtained b}^ replacing integrals w^itli sum- 
mations and differential equations with difference 
equations. 

4. Estimation of the Mean 

Let [X(t) ] be a contniuous-time woakh" stationary 
process with the mean ju, the covariance function 
7(.s'), and the spectral density function gij). Let 
a sample function X{t), 0<t<T,he available. Con- 
sider the sample mean 



7n^ 



Jo 



X{t)dt. 



(4.1) 



We have Em — ^i, and [Siddiqui, 1961, eq (2.9)] the 
variance of m. 



var m= 



2T-'r (l-slT)y(s)ds. (4.2) 



The variance of 7n can also be expressed in terms of 
g{J). In fact, using (3.5b) in (4.2) and interchanging 
the order of integration with respect to / and s, we 
obtain 



var m=2 



r°° sin' 

Jo (tt: 



"^gimf. 



Tf) 



(4.3) 



Changing the variable of integration in (4.3) by 
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setting /'=7rJy", we have 



var 7n- 



where again f is replaced by /. Expressions (4.2) 
to (4.4) are exact. However, if g(f) is continuous 
at f— and g(0)>0, an asymptotic expression for 
var m is 

.9(0). 



var m^ 



4/; 



j{s)ds. 



(4.5) 



Thus when g(f) is continuous at zero and (i)g 
(0)>0, var m = 0(T-'), (ii)g(0)=0, var m=0(jr-i). 
In any case, var m->0 as T-^oo whenever g(f) is 
continuous at zero, so that m tends to /x in proba- 
bihty (ergodic property). 

For a discrete sample Xj,=X(kh), k=l, 2, . . ., N, 
the corresponding results are 



var 7n = 



i = l 

giio) 



N 



(4.1d) 
(4.2d) 
(4.5d) 



Here, yk=y(kh) and (4.5d) holds when tji(f) is 
continuous at zero. 

Let N' be defined by the equation 



iV' =7(0) /var m, 



(4.6) 



where var m is given by (4.2) or (4. 2d). N^ will be 
called the equivalent random sample size for estimating 
the mean, as the variance of the mean of A^^ uncor- 
related observations is y{0)/N\ 

Example 4.1. Let 7(.^)=^(7V.-^l^lcos 27r/oS, X>0. 
This type of covariance function occurs sometimes 
in the analysis of radio propagation data. For a 
detailed discussion of it see cliapter 5 of Bendat 
[1958]. Since \y{s)\ is integrable, g{f) is continuous 
everywhere. We have 



</(0) = 



'J: 



^^ cos 2Tfosds= 



2a^\ 



\' + ^T'fo 



Hence, forXT>>l, 



var m^ 



2X(7- 

T(X^+47ry?)' 



Of course the exact var m can be calculated from 
(4.2). If the sample is discrete, setting p=e~^^j 
where h is the sampling interval, we find 



var m^ 



cy\l-p') 



iV(l + p'-2pcos27r/oXA) 

We may remark here that y{s) is not differentiable 
at s==0, hence the process is not differentiable. 



Example 4.2. Let y(s) = (T^e-^l'\, X>0. This is a 
special case of Example 4.1, when/o=0. Thus 

2a' (7-(l+p) 

var m^^ or ^^^^^y 

as the case may be. If X = 0. 69315, and h=l, the 
p=0.5, and the equivalent random sample size A^' 
is approximately 0.35 T or 0.33A', respectively. 
Again, y(s), hence the process, is not differentiable. 
Example 4-^- Let y{s) = (T' cos 27r/o.9, /of^O. Note 
that this cannot be considered as a special case of 
Example 4.1, as y{s) is not integrable. This is a 
case when g{f) does not exist. However, we can 
either use (4.2), or (4.3) with g{f)df replaced by 
dG(f), where dG(f) = l/2a', if f=±fo; 0, otherwise. 
Thus 



var m- 



, sm^ T 



Tfo 



{^Tf,y 



Thus var m=0(T-') instead of 0(T-'). Furtliermore, 
if T=k/fo, k a positive integer, var m = 0. 
Example 44- Let 



g(f)= 



a\l-a) _1_ 



2B' 



0<a<l,if|/|<Zi;-0,if|/!>5. 



This type of spectral density function has been 
observed for the frequency fluctuations of the 
received signal, when the transmitted signal has 
constant frequency. Since g(J) is specified instead 
of y(s) it is more convenient to use (4.3). Also, 
since g(J) is not continuous at /=0, asymptotic 
approximation cannot be used. Setting tzTB=A, 
we find 



var 



0-^(1— a) f"^ sin^x , 



The integral can be evaluated mmierically. An 
upper bound to var m is obtained by donhnating 
sin^ X by x^ when 0<x<l, and by 1 when a;>l. 
Thus, if ^<1, var m<^a^', and, if ^>1, 



var m< 



2ct' 



a\l-a) 



{l+a)A'-' {l+a)A' 



Thus var m=0(r-^+«) rather than 0(T-'), Note 
that /^^ g(f) is integrable for p = l, 2, . . ., hence 
the process is differentiable to any order. 

5. Estimation of the Covariance and the 
Spectral Density Functions 

In this section we will confine ourselves to discrete- 
time Gaussian processes. 

Let {Xfc}, k= . . ., — 1, 0, 1, . . ., be a discrete- 
time Gaussian stationary process with mean ju, 
covariance function 7^, and spectral density g(J). 
g{f) and 7;^ are related by the transform pair 



-=^r 



cos C2xfk)g{J)dj, k =0,±1,±2, . . ., 
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<7C/)=To+2 St*cos (27r/fc) -1/2</<1/2. (5.1) 

k=l 

Let N consecutive observations be made on the 
process. We may denote them as Xi, X2, . . ., 
X^. From the sample we calculate 

m=N-'T>X„Cs={N-s)-'^iX,-m){X,+ -m), 

k=l k=l 

(5.2) 

for s=0, I, 2, . . ., n, where 72-^00 as N-^^ but 
njN^O. Thus, for example, n may be the largest 
integer in N^^\ If 



S yfc=(iyo< «> ; S yl=byl< ^ ; 

k=l k=l 



it can be shown that 



EC.'. 



7Q(l+2a) 

— at-^ 



(5.3) 



var Cs^N-' [7?+7:+2 g (tI+7.-s7.+.)] • (5.4) 
Thus Cs is biased as an estimate of 7, but the bias, 



and also the variance, tend to zero as A^- 



Thus 



Cs-^ys in probability (ergodic property). We note 
that if \y]c\ is summable, both 7^ and 7I will be 
summable (since 7K7o|7a|), and the ergodic property 
will hold. 

We note in particular that the estimate of the 
variance 70 is Cq. From (5.4) 

^ 27^(1 + 26) , - ,_ ._, 

var Co^ ^-j^ ^; 6=2 7I-/75. (5.5) 

The equivalent number of degrees oj freedom for 
estimating the variance may be defined as 



var Co 



['-^]' 



N 



75(1+26) -(1+26) 



Example 5.1. Let y^=a^p\''\, |p|<l. Then 6 = ^^ 
/(1-p^), 

and 

7V(l^p^. 

= 1+p^ 

Thus, if p= ±0.5, 7V'^3A^/o; if p- dzO.9, A^'^A^/10- 
Exam-pie 5.2. Let y,:=a^e~^''' , X^O. Then 

1+26=S e-2«'^[7^/(2X)]"^ if X<1, 

— 00 

2a' /^\'/2 



var Co? 



A^ 



(i) 



A^' 



--(?: 



1/2 



J 



Thus, if 6-^=0.5, A^'^0.667V;; if 67^-0.9, N'^0.26N. 
We turn now to the estimation of the spectral 
density function. The classical harmonic analysis 
leads to the periodograni estimate. 



On 



(jl}=^(<^J + b^)J-h^. . . viV-1, 



where N is assumed to be an odd integer, and where 
2 ^ ^ 2Trjk. , 2 ^ ^ . 2Trjk 



' NU 



~NU 



N 



It can be shown that if j/N-^f as N^co ^ Hm Eg^^ < 

U/N) = g(f), var gN(f)^gHf); so that g^,(f) does not 
converge to g(f) as iV— >oo. We mention here two 
alternative estimates which converge in probability 
to g(J), i.e., which are consistent estimates of g(f), 
(1) Bartlett [1950]. 

^^^ (/)=!] (l-f)a.cos27r/:/. 



where n-^00 as N^^ but n/N^O. It is found that 
lim Eg^^Kf)=g(f), 



iV-^a 



^^9'(f)J^0, 



4:n 



var (/^P(/)^^ 

(2) Blackman and Tukey [1958]. 

g'i^\f)=^Co+2 § (^0.46 cos ^+0.54) a cos irkf 

+ (0.46cos7r+0.54)(7^ 

cosTrnfJ=j/n,j=0,l, . . .,n-l; 

lim Eg^\f)=g{f), 



Y^rg^j^\f)^^ 



0.8j(/^(/),when/^0, 



1.6^^2(0), when /=0. 



Parzen [1957] discusses a general method of obtaining 
consistent estimates of the spectral density function. 
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Another powerful tecliiiique is to find a linear 
filter W(t) such that if Xt is the input, the output, Yty 
is approxiniately a white noise, i.e., gy(f) = crl, — 1</ 
<|. The variance, al, of Yf can be estimated 



consistently, hence, (JxiJ)'- 
estimated consistently. 
Examfle 5.3. Consider 



|M'*(/)po-L <^"^ii ^^^^ be 



Here, 



Xt=aoYt+aiYt_^+ . . . +apYt-T,. 



W{t) =at, if ^ = 0, 1, . . ., 2^=0, otherwise. 

Replacing integrals by summations in (3.9) and 
(3.10), we obtain 



g.U)= 



2 a,6-2-^-^^ 

^=0 



9vU). 



If Yt is a white noise, A^^ is called a rnoving average 
process. On tlie other hand, if Xt is a white noise 
and 



k={) 



-0 



has all the roots within the unit circle \Z\ = 1 in the 
complex plane, Yt is a stationary process and is 
called an autoregressive process of order p. In the 
former case gy(f) = (rl, and in the latter case gx(f) = 

Special case {a) . li ak=ll{p+l),k=^,l, . . ., p, 
and Yt a white noise, Xt is called a simple moving 
average pfrocess. In this case gxij) is evaluated to be 

^-y^-^+1 sinM^/) '' 2SJS2, 

as o-^=(^ + l)cr|. We only need a consistent 
estimate of al, wliich is the sample variance. 

Special case (6). Yt—pYt-i=Xt, |p|<l, a.nd Xt a 
white noise. This is a first-order autoregressive 
scheme. Setting aQ=l, ai = —p, and ^^=0 otherwise, 
we find 

g^{f) = {l-2p cos 2x/+pO-V.(/). 



Also 
hence 



gx{f) = (Tl=al{l-p'); 



gyU)=^al{l-p'){l-2pcos2Tf + p'y 



al and p are consistently estimated by the sample 
variance and the first (lag 1) autocorrelation (=Ci/ 
Co), respectivel.y. Note that the covariance of 

{Yt}, yy{k) = (jlp^^K Hence, gy{j) can also be 
obtained directly from 

gyU) = <rl [l + g (pV-^^^+P^6— •^^)]. 

Example 5. If. In figure 1, 236 noon hour monthly 
median /o^2 values in Mc/s observed at Washington, 



D.C., from May 1934 to December 1953 are plotted 
against t. The unit of t is one month and t=l cor- 
responds to January 1934. Hence, the first value 
plotted corresponds to ^=5. In figure 2 the autocor- 
relation function, r(r) = 6V/(7o, of this data is plotted 
for T=\, 2, . . ., 120; and in figure 3, the Black- 
man-Tukey spectral density function, gN^(f), is 
graphed against frequency, /, cycles per year. 
QnU) is the normalized density per cycle per year. 
The spectral density indicates that there are two 
fundamental cycles in the data corresponding to 
/=0.1 (10-3^* cycle), and/=l (one yr cycle). Be- 
sides these, their first two harmonics (y=0.2, 0.3, and 
/=2, 3), and their ''interaction'' frequencies {f—l 
±0.1 = 0.9, 1.1) are also significant. Since the 
least common period for all these cycles is 120 
months, before further analysis it seemed advisable 
to add four more terms to the data in figure 1, to 
make the total number of data points 240 = 2X120. 
These values are 5.9, 5.7, 5.5, and 5.0 corresponding 
to the months of January-April 1954. 
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MEAN OF THE DATA = 8.12 
VARIANCE OF THE DATA = 5.92 
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Figure 1. Noon-hour inonthly median UYo^-iMcIs) at Wash- 
ington, D.C., from May 1934 io December 1953, 
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Figure 2. Autocorrelation function of the data in figure 1. 
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Figure 3. Spectral densitij function of the data in figure 1. 

Let x(t) denote the /oi^2 value at time t. Then 
x{t) is represented as 



where 



m 



x{t) = m{t)+2[t), 

(O^X+S (^'; cos -]|f +^-sm ^Ij; 
t 



and the summation over j is on j = l, 2, 3, 9, 10, 11, 
20, and 30. z{t) represents the ''error'' or ''noise/' 
The coefficients a^ and hj are obtained by the least- 
squares method, and their values are as follows: 

Table 1. a,-, hj, and A? + 6- 



Table 2 



; 


ttj 


bi 


a^+bf 


1 


-2.24 


1.17 


6.38 


2 


-0. 371 


-0.297 


0.226 


3 


-.0172 


-.220 


.0485 


9 


-.130 


.393 


.171 


10 


-.671 


-1.77 


3.58 


11 


.342 


0.443 


0.313 


20 


.556 


.0556 


.313 


30 


-.233 


-.247 


.115 



After fitting m{t), the residuals 2{t) are calculated 
from z{t)=x{t)—m(t). Autocorrelation analysis of 
z{t) shows that they can be considered as "white 
noise.'' To test whether the residuals are normally 
distributed, the range (— <», «>) is divided into 10 
intervals such that in each interval the expected 
frequency is the same, i.e., 24. For this purpose an 
estimate of var 2 is required. This estimate is 

^' N-17 223 ^-^ • 

We note that ^2(t) = 0. The resulting class inter- 
vals and the observed frequencies, /o, are tabulated 
in table 2. 



Class interval 


/o 


— 00 


. . -0.801 


23 


-0. 801 . 


. . - .526 


20 


- .526 . 


. . - .327 


25 


- .327 . 


. . - .158 


24 


- .158 . 





26 





.158 


24 


0.158 . 


0.327 


33 


.327 . 


.526 


20 


.526 . 


.801 


20 


.801 . 


.00 


25 



The expected frequency, /g, for each class interval 



is 24. Hence x2==2 



This value 



/. 24- 

is calculated to be 5.67, and the number of degrees of 
freedom for x^ is 8 . The probability that such a sample or 
worse comes from a normal distribution, as judged 
by the x^ value, is more than 60 percent. We may 
thus conclude that we have essentiall}^ completed 
our analysis and that m(t) is the best fit to the data. 
The correlation coefficient between x{t) and m{t) 
is given by 



R-- 



<-rK^ 



Q.39Y 
"5.96/ 



^0.97 



m(t) is plotted against t in .figure 4 for ^=1, 2, . . ., 
120. The values oi m(t) for t=l, 2, . . ., 12 may 
be taken as predictions for the successive months 
of a year ending in 4, i.e., 1954, 1964, 1974, the 
values for ^=13 to 24 for the months of a year 
ending in 5; and so on. Since the residual standard 
deviation is 0.62, uniform 95 percent confidence 
limits for x(t) are m(0dzl.2. 
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Figure 4. Prediction curve for monthly inedian WF2 for Wash- 
ington, D.C. 
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6. Estimation of the Distribution Function 

The process (listribiition fuiictiou, P(;r), is the 
probabiht}^ that Xt<x. The samph^ distribution 
function, P*{x), is the proportion of JVi, . . ., X^, 
such that Xr<x. Thus, with ^=l, 2, . . ., N, 



P*(^) = 



fO, if ^<all A",, 
j/N, if exactly j of Xi < x, 
l,if all.Y,<a:. 



(6.1) 



We will assume {Xi} to be strictly statioimry 
process and inti'oduce 



PM ,x.2) =Pr{X, < X, , X,+, < x.) 
=Pr{Xo<x,, X,<X2). 

Note that, il' ;r2>/i 

Po(xuX-2)=Pr{X,<Xr,X,<x.2) 

=Pr(X,<x,)=P(x,). 

r,= ],if A',</;(). if.Y,>:f; 



(6.2) 



Let 
then 



7>*(,,)=^v-'z:i^; 



(6.3) 
(6.4) 

(6.5) 



i.e., P*{x) is a sample mean of the {Y,} process and 
the results of section 3 will apply, noting that 

EY,=P>iX,<x)=Pix) 

y,{s)=ElY-P(xy\\Y,+ -P{x)] 

=P>iX,<x,X, + ,<x)-P\x) 

=P,{x,x)-P\x). (6.6) 

Since P(,ix,x)=P{x), we have 

y,m=nr)\l-P(x)\. (6.7) 

I'hus, from (4.2(1), 

var p*(:c)=^+| g (l-|) yAk), 

where yy(k) are given in (6.6) and (6.7). It is 
obvious that if 

S7.(^)=^S [P,(x,x)-PHx)]<<^, 

k=l k=l 

then var P*(x)^0 as iV— >«>, and P*(x) converges 
in probability to P{x) (ergodic property). 

To estimate var P'^ix) from the sample, we need 
an estimate of Pk(x,x). For this purpose we find 
the proportion of the sample pairs (.Y,, Xt+k) such 
that both Xt<x and Xt+}:<x. We will denote this 
proportion as Pl(x,x). 



Example 6.1 . The following is a systematic sample 
of 80 observations (read at 5-sec intervals) of received 
field intensity in (microvolts)^. Read left to right. 

0.20, 0.71, 0.06, 0.05, 0.76, 0.32, 0.96, 

0.63, 0.09, 0.18. 0.25, 0.45, 0.26, 0.10, 

0.95, 0.01, 0.50, 1.20, 1.99, 0.32, 0.51, 

0.01, 0.16, 0.56, 3.16, 1.27, 2.24, 1.00, 

0.81, 1.29, 0.28, 0.21, 0.35, 0.20, 0.39, 

0.89, 1.24, 0.08, 0.98, 1.01, 0.49, 0.90, 

1.90, 1.42, 1.56, 1.32, 1.20, 1.59, 2.40, 

2.24, 0.80, 0.56, 1.45, 0.18, 0.02, 0.28, 

0.81, 0.18, 1.13, 0.64, 1.95, 0.48, 0.55, 

0.44, 0.28, 0.07, 0.71, 0.48, 0.40, 0.06, 

0.79, 1.01, 0.51, 0.70, 0.14, 0.16, 0.01, 

0.06, 0.03, 0.01. 

Let us consider tlie estimate of 7^(0.5). We obtain 
7^*(0.5) = |^ = 0.4875; 

Pi*(0.5,0.5) = ~ = 0.2911,A*(0.5, 0.5) = |^ = 0.2692; 



P3*(0.5, 0.5) = ^-0.2597, A*(0.5, 0.5)-^ = 



= 0.2105. 



Noting that 

P-*(0.5) = 0.2377, 7,(0)=P*(0.5)[1-P*(0.5)] = 0.2498, 

where yy{k) denotes the sample estimate of yy{k), we 
find that 7^/(3) and 7^(4) are of opposite sign and 
7i/(3)/rv(0), 7v(4)/t2/(0) are negligible compared to 

unity. We may assume that X] Jyi^) is negligible. 

Thus 

var P*(0.5)^^,-|[(l-^) (0.0534) 

-(l-|)0.0315] 



+ 



-0.0031+0.0021 

-0.0052. 

With the assumption of approximate normality for 
the distribution of P(0.5), 95 percent confidence 
limits for P(0.5) are 



P*(0.5)-1.96.9^<P(0.5)<P*(0.5)-fl.96s,„ 



where 



s^=VvarP*(0.5) =0.072, i.e., 0.35<P(0.5)<0.63. 

If we assume the Rayleigh power distribution for 
the above data, the distribution function is most 
efliciently estimated as 
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P{x)-- 



-l- 



0-71 0<a:< 00. 



where 0.71 is the mean of the observations. Thus, 
under tlie assumption of the Rayleigh distribution, 
P(0.5) is estimated to be 



Ao.5) = 



0.50 

"^•^1=0.51. 
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